WORLD INTELLECTUAL PROPERTY ORGANIZATION 
Internationa! Bureau 




PCT 

INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(51) International Patent Classification 6 : 

G06K 9/00, 9703, G10L 3/02, G06F 15/00 



Al 



(11) International Publication Number: WO 97/18526 

(43) International Publication Date: 22 May 1997 (22.05.97) 



(21) International Application Number: PCT/US967 1 8220 

(22) International Filing Date: 12 November 1996 (12.1 1.96) 



(30) Priority Data: 

08/557.994 



1 3 November 1995 (13.11 .95) US 



(71) Applicant: MOTOROLA INC. 1US/US); 1303 East Algonquin 

Road, Schaumburg. IL 60196 (US). 

(72) Inventors: LEE. Yenwei; 3041 Calle De Las Flores, San Jose, 

CA 95148 (US). DONG. Yu; 613 Escuela Avenue #3. 
Mountain View. CA 94040 (US). 

(74) Agents: SARLI, Anthony. J. et al.; Motorola Inc.. Intellectual 
Property Dept.. 1303 East Algonquin Road, Schaumburg, IL 
60196 (US). 



(81) Designated State: CN. 



Published 

With international search report. 



(54) Title: METHOD AND APPARATUS FOR CHARACTER RECOGNITION INTERFACE 



(57) Abstract 

A method and apparatus for character recognition interface is 
disclosed where input data (102) received from a writing area of 
a display (210) is compared with character recognition data (220). 
At least one likely candidate character representing a word likely to 
represent the input data is generated (230) and displayed (240). In a 
preferred embodiment, at leas! one candidate character representing 
an alternative word likely to represent the input data is generated and 
displayed (225). At least one series of correlating candidate characters 
for the at least one likely candidate character and for each alternative 
candidate character is generated and displayed (370). Each series 
of correlating candidate characters represents a series of correlating 
words likely to include a word representing the input data. The likely 
displaying candidate character is displayed in an output buffer area 
(385) of the display. Each alternative candidate character is displayed 
in a candidate list area of the display. Each character displayed can 
be stored in a memory. 
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METHOD AND APPARATUS FOR CHARACTER RECOGNITION 

INTERFACE 

Field of the Invention 

5 

This invention relates generally to data input interfaces, 
and more particularly to character recognition interfaces of 
large characters sets where each character includes one or 
more strokes. 

1 0 

Background of the Invention 

Machine recognition of human handwriting is a very 
difficult problem, and with the recent explosion of pen-based 
1 5 computing and electronic devices, has become an important 
problem to be addressed. There exists various different 
computing and electronic devices that accept handwritten input. 
So called pen-based products, for example, computers, and 
personal digital assistants, and the like typically have a touch 

2 0 sensitive screen upon which a user can impose handwriting. 

These devices then function to digitize the handwritten input. 
Other devices, such as computers, advanced telephones, digital 
televisions, and other information processing devices, include or 
can access a digitizing tablet which can accept handwritten 

2 5 input. Still other devices can receive handwritten character 

input by means of a fax, scanned input, electronic mail, or other 
electronic transmission of data. These devices process the 
information and attempt to recognize the information content of 
the handwritten input; typically, the device then displays that 

3 0 information to the user for purposes of feedback, correction of 

errors in the processing, and for recognition of the handwritten 
character input. 

There exists various approaches for recognition of 
handwritten input when the recognition is for characters sets 
3 5 having a limited finite number of characters, typically under a 
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hundred. However often such approaches do not work as well 
for character sets having large numbers of varied complex 
characters. Examples of large character sets that have been 
difficult to quickly and accurately recognize through 
5 recognition of handwritten input are several of the Asian 
ideograghic character/symbol languages, such as Chinese, 
simplified and traditional, Japanese, and other languages 
having large character sets. Some languages such as simplified 
Chinese consist of several thousand characters. 
1 o Traditional interface methods, such as keyboard entry, of 

inputting data and text supplied in one of these types of large 
character based languages is often very difficult; in part 
because of the large number and complexity of the character 
set. Additionally, many of these languages resort to phonetic 

1 5 based representations using Western characters in order to 

enter the characters with a keyboard. Hence, keyboard-type 
entry of such characters is difficult. An example of the 
difficulty of keyboard entry for a large character set based 
language is keyboard entry of the Chinese language. To enter 

2 0 data, or text, in Chinese, via a keyboard, the language is first 

Romanized. Western Characters, such as the English anglo- 
saxon alphabet, are used to phonetically represent the 
characters of the Chinese language. This is referred to as Pin- 
yin. Therefore, for a person wishing to enter data or text in 

2 5 Chinese through a keyboard, the person must first know Pin- 

yin, and the corresponding English character representation for 
the phonetic equivalent of the Chinese character they are 
trying to enter via the keyboard. 

Another, difficulty encountered with traditional keyboard 

3 0 interface input for recognition of handwritten input of data, or 

text, based upon a language having a large character set is 
many of these languages have one or more forms of 
representing the same character, i.e., similar to print and 
cursive forms for the English angle-saxon alphabet. 
3 5 Additionally, such languages may have homophones for 
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example, the Chinese language has numerous homophones - 
words that are pronounced the same but have different 
meaning and written forms. Hence, the same Pin-yin can refer 
to a multiplicity of characters and the person entering Chinese 
5 character data must often select from a list of possible choices. 
Typically, interface techniques used for data input of 
limited finite character sets of under hundred, do not produce 
accurate results for languages having large character sets, of 
several hundred or several thousand varied complex 
1 0 characters. Many of the interface techniques used for data 
input such as handwriting recognition of small character set 
languages are very slow when used for large character set 
languages. 

Therefore, because of the increasing use of pen-based 

1 5 electronic input devices, the difficulty of keyboard entry for 

various languages, including large, complex, character set 
languages, a need exists for a method and apparatus for 
character recognition interface of handwritten input for various 
languages, complex, large character set languages, symbols, and 

2 0 numerals that is quick, accurate, and easy to use. 

Brief Description of the Drawings 

FIG. 1 Illustrates a perspective view of an apparatus of a 

2 5 preferred embodiment of the present invention. 

FIG. 2 Illustrates a perspective view of an alternative 
apparatus of a preferred embodiment of the present invention. 

FIG. 3 Illustrates a preferred method of character 
recognition interface of the present invention. 

3 0 FIG. 4 Illustrates a flow diagram of operation of a 

preferred embodiment of the present invention. 

FIG. 5. Illustrates the remaining portion of the flow 
diagram of operation of FIG. 4. 

FIG. 6 Illustrates graphically a preferred method of 
3 5 character recognition interface having a language model phase. 



WO 97/18526 



PCT/US96/18220 



-4- 

FIG. 7 Illustrates graphically an alternative preferred 
method of character recognition interface having a language 
model phase. 

FIG. 8 Illustrates a flow diagram of operation detailing a 
5 preferred embodiment of character recognition interface 

having language model phase in accordance with the present 
invention. 

FIG. 9 Illustrates the remaining portion of the flow 
diagram of operation detailing a preferred embodiment of 

1 0 character recognition interface having language model phase in 

accordance with the present invention illustrated in FIG. 7. 

FIG. 10. Illustrates graphically a preferred method of 
character recognition interface having an editing output buffer. 

FIG. 11. Illustrates a flow diagram of operation detailing 
15 a preferred embodiment of character recognition interface 

having an editing output buffer in accordance with the present 
invention. 

FIG. 12. Illustrates graphically a preferred method of 
character recognition interface having an on screen keyboard 

2 0 FIG. 13. Illustrates a flow diagram of operation detailing 

a preferred embodiment of character recognition interface 
having an on screen keyboard in accordance with the present 
invention. 

FIG. 14. Illustrates graphically a preferred method of 

2 5 character recognition interface having a plurality of user 

adaptable features 

FIG. 15. Illustrates a flow diagram of operation detailing 
a preferred embodiment of character recognition interface 
having a plurality of user adaptable features in accordance 

3 0 with the present invention. 

FIG. 16 Illustrates graphically a preferred method of 
character recognition interface having a plurality of user 
adaptable features 

FIG. 17 Illustrates a flow diagram of operation detailing a 
3 5 preferred embodiment of character recognition interface 
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having a plurality of user adaptable features in accordance 
with the present invention. 

FIG. 18 Illustrates an alternative interface in accordance 

with the present invention. 
5 FIG. 19 Illustrates a flow diagram of the alternative 

interface of FIG. 18 in accordance with a preferred method of 
the present invention. 

FIG. 20 Illustrates the remaining portion of the flow 
diagram in FIG. 19 of the alternative interface of FIG. 18 in 

1 0 accordance with a preferred method of the present invention. 

Detailed Description of Preferred Embodiment 

Generally, the present invention relates to a method and 
15 apparatus for character recognition interface of handwritten 

input; and preferably the present invention relates to a method 
and apparatus for character recognition interface of 
handwritten input representing one or more characters 
selected from a language or compilation of data having a large 

2 0 complex set of characters where each character includes one or 

more strokes. Generally, an alternative embodiment of the 
present invention relates to a method and apparatus for 
character recognition interface of handwritten input 
representing a graphical image of a signature or other such 

2 5 identifying set of characters. 

Pursuant to a preferred embodiment of the present 
invention, candidate characters in support of a handwriting 
recognition method and apparatus of the present invention are 
developed through the compilation and statistical analysis of 

3 0 empirical data compiled from hundreds of samples of actual 

handwritten characters. Candidate characters produced 
through the development of templates derived from the 
compilation and statistical analysis of the empirical data are 
selectable as the recognized character of the handwritten 
3 5 input. 
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Referring now to the Figures, FIGS. 1 and 2 illustrate the 
general operation of a method and apparatus in accordance 
with a preferred embodiment of the present invention. With 
reference to FIGS. 1 and 2, examples of pen-based electronic 
5 entry devices are illustrated. A personal digital assistant is 
illustrated in FIG. 1 as generally depicted by the reference 
numeral 10. The personal digital assistant (10) depicted 
constitutes a generic representation, typically such devices 
include a housing (12) and a touch screen (18) upon which 

1 0 input can be handwritten using an appropriate hand 

manipulated stylus (15). Such devices typically include one or 
more microprocessors or other digital processing devices. As 
such, these devices comprise computational platforms that can 
be readily adapted in accordance with the teachings presented 

15 herein. Generally, such devices include one or more application 
programs, or target applications, operating under control of an 
operating system comprising instructions for the 
microprocessor or digital processing device. It should be 
understood that, while such personal digital assistants comprise 

2 0 a ready platform to accommodate the practice of the applicant's 
teachings, the teachings presented herein may be practiced in a 
variety of other operating environments as well. Some 
examples of such environments include, but are not limited to 
the following, computers or other electronic entry devices with 

2 5 digitizing screens, or connected to a digitizing input surface, or 

capable of receiving faxed, scanned, or other electronic input, 
digital or interactive televisions, modems, telephones, pagers as 
illustrated in FIG. 2, or other systems with the ability to 
capture handwritten input and process it. The references 

3 0 illustrated in FIG.. 2 refer to substantially similar elements as 

illustrated in FIG. 1. on an alternative device as illustrated in 
FIG. 2, i.e., device 20, touch screen 28, housing 22, stylus 25 
Referring now to FIGS. 3, 4, and 5, FIG. 3 illustrates a 
preferred method of character recognition interface of the 
3 5 present invention and FIGS. 4 and 5 illustrate a flow diagram of 
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30 
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operation corresponding to FIG. 3. Typically a character is 
input into a device (100) having a memory by means of a 
stylus (115) or like device on a digitizing surface (117). In 
accordance with the present invention the input character 
(102) is mapped to a writing surface (100) or area on the host 
device (100) through the digitizing surface (117). Various 
known technology exists for mapping strokes to a selected 
region of a display area. The input data is compared with 
recognition data (220). The recognition data is obtained by 
accessing a recognizer designed to generate a least one most 
likely candidate character to represent the input character. 
Generally, the input character is preprocessed into a series of 
strokes and is processed by the recognizer by comparing the 
series of strokes with stored recognition data or templates to 
eenerate at least one most likely corresponding candidate 
character. In accordance with the present invention various 
recognizer and recognition data may be used. 

In a preferred embodiment of the present invention the 
recognizer and recognition data are configuring to recognize 
and provide recognition data for the Chinese characters 
comprising the sets of GB1 and GB2. Additionally, a preferred 
recognizer and recognition data for the present invention are 
configured to recognize and provide recognition data for the 
Chinese character of GB1 and GB2 and the Taiwanese characters 
of Big 5-1 and Big 5-2. In a preferred embodiment of the 
present invention the recognition data generates a graphical 
image of at least one likely corresponding candidate character 
(230). If no such image can be generated by this comparison 
with the recognition data, a graphical image of the 
unrecognized input character can be stored in memory (232), in 
the output buffer (116) and sent to an application resident and 
operating in the device (235), or sent directly to an application. 
Alternatively the input character may be deleted (238). If at 
least one likely corresponding candidate character is generated 
by comparison with the recognition data, typically, a first most 
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likely candidate character is displayed (240). This candidate 
character can be displayed in various areas of the display 
screen; the writing area (110); the output buffer area (116); the 
candidate character list area (112); and the language model list 
5 area (114). Preferably, the first most likely candidate 

character is displayed in the output buffer area of the display 
(240). In accordance with the preferred invention comparing 
the input character with the stored recognition data can 
generate at least one likely alternative candidate character in 
1 0 addition to the likely candidate character. Further, in 

accordance with the preferred embodiment of the present 
invention, the number of alternative candidate characters 
generated and displayed can be set by the chosen recognizer; 
the user; or a default maximum. In a preferred embodiment 

1 5 the default maximum does not exceed ten for the number 

generated of likely alternative candidate characters. In 
accordance with the present invention various recognizer and 
recognition data, or templates, may be used. In a preferred 
embodiment of the present invention the recognition data 

2 0 generates a graphical image of at least one likely corresponding 

candidate character and a graphical image of at least one likely 
alternative candidate character. In a preferred embodiment 
the likely candidate character and the likely alternative 
candidate character can be displayed in various areas of the 

2 5 digitizing surface (117) or display screen (118); the writing 

area (110); the output buffer area (116); the candidate 
character list area (112); and the language model list area 
(118). If the most likely candidate character is not an accurate 
graphical representation of the input character (245), then at 

3 0 least one alternative corresponding candidate character is 

displayed if one is generated by the comparison of the input 
data with the recognition data (250). If no alternative 
correlating candidate character is generated that accurately 
represents the graphical image of the input character (250) 
3 5 then the user can select from at least one of the following 
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options: i) store the unrecognized input character (257) and 
send to a target application (259), ii) delete input (256), iii) 
input a new character (258). If an alternative corresponding 
candidate character exists, preferably the alternative candidate 
5 character representing the input is selected (260) replacing the 
first likely candidate character (265). In accordance with a 
preferred embodiment the selected alternative corresponding 
candidate character is stored or displayed in the output buffer 
display area (270). Preferably the alternative corresponding 
10 candidate can be sent directly to a target application, or sent to 
the target application from the output buffer (275). 

Referring now to FIGS. 6, 7. 8, and 9, FIGS. 6 & 7 illustrate 
alternative preferred methods of character recognition 
interface of the present invention having language modeling; 

1 5 and FIGS. 8 & 9 illustrate a flow diagram of operation 

corresponding generally to FIGS. 6 & 7. In accordance with the 
preferred embodiment of the present invention, the feature, or 
option, of language modeling may be accessed through a 
plurality of methods. Language modeling for the purposes of 

2 0 the present invention is the combination of a series or string of 

characters, word, or symbols that have a likelihood of being 
associated or correlated to one another to form a phrase, 
mathematical expression, or other symbolic expression 
consisting of a plurality of characters. An example of a 

2 5 language model series or string of correlating candidate 

characters that may be generated from comparison of input 
data with an English language based recognizer and recognition 
data is the phrase "Thank God it's Friday". For purposes of the 
present invention the term character referred to throughout 

3 0 may mean a single letter, a word, one or more ideographic 

characters, symbols, numbers and the like. Further in 
accordance with the preferred embodiment of the present 
invention, the number of series of correlating candidate 
characters generated and displayed can be set by the chosen 
3 5 recognizer; the user; or a default maximum. In a preferred 
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embodiment the default maximum does not exceed ten for the 
number of series of correlating candidate characters generated 
to include the likely candidate character or generated to 
include each alternative likely candidate character. 
5 A toolbar language model option as illustrated in FIG. 6 

permits the language modeling feature to be activated. 
Alternatively, the language modeling feature may be displayed 
in a language modeling area of the display as illustrated in FIG. 
7 and be activated from a mouse, stylus or other input device. 
10 In accordance with a preferred embodiment of the present 
invention, the toolbar of the present invention may be 
selectively enabled and disabled. Additionally, the toolbar may 
be selectively placed, or displayed, on various areas of the 
display. Preferably the toolbar is displayed above or below the 

1 5 writing area in some manner. 

When the language modeling feature of the present 
invention is activated the input character (102) is compared 
with the recognition data to generate at least one series of a 
string of correlating candidate characters (330), where such 

2 0 series includes the graphical representation of the character 

input and a plurality of characters that are commonly 
combined and include the input character. For example, if a 
character input is the word "Friday", a language modeling series 
of correlating candidate characters generated by comparison 

2 5 with the recognition data may generate the phrase "Thank God 

its Friday". Preferably alternative series of strings of 
correlated candidate characters are also generated from which 
selection of an accurate series containing the character input 
may be made. The term correlated for purposes of the present 

3 0 invention means associated with, commonly combined together, 

or the like characters commonly associated, or combined 
together that include the input character. 

In accordance with a preferred embodiment of language 
modeling of the present invention, an input character is 
3 5 received and mapped into a writing area of a display (310). 
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The input character is compared with recognition data (320). If 
the comparison with the recognition data does not generate at 
least one likely series of correlating candidate characters (330) 
various options can be selected including the following: i) store 
the unrecognized ink of the input character (332) and send to 
the target application (335), ii) delete the ink of the input 
charter (336), and iii) input a new character (338). In 
accordance with the present invention ink includes the 
digitized representation of the input character, the raw input 
character, regardless if the character is handwritten, scanned, 
or entered by some other means. 

In a preferred embodiment, if the comparison with the 
recognition data does generate at least one likely series of 
correlating candidate characters (330) then the first, or most 
likely, series, or string, of correlating candidate characters is 
displayed (340). Preferably the series is displayed in a 
correlating candidate character area of the display. The 
correlating candidate character area of the display allows the 
series of correlating candidate characters to the selectively 
displayed from a language model list area (114) of the display, 
entered and displayed in the output buffer area (116) of the 
display, or be displayed by selection of the language model 
option on the toolbar (108). If the displayed first, or most 
likely series of correlating candidate characters displayed 

2 5 accurately represents a string, or phrase of characters 

" containing the graphical representation of the input character 
(350) then preferably the string is stored in the output buffer 
(355) and sent to the target application (358). However, if the 
first most likely, series of correlating candidate characters does 

3 0 not accurately represent a string, or phrase of characters 

" ' containing the graphical representation of the input character 
(350) then at least one alternative series of likely correlating 
candidate characters is displayed (360). If no alternative series 
of likely correlating candidate characters is generated by 

3 5 comparison of the input data with the recognition data that 
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accurately includes a graphical representation of the input data 
(370) then various options can be selected including the 
following: i) store the unrecognized ink of the input character 
(372) and send to the target application (375), ii) delete the ink 
5 of the input character (376), and iii) input a new character 
(378). 

If the comparison with the recognition data does generate 
at least one likely series of alternative correlating candidate 
characters which accurately includes a graphical representation 

1 0 of the input data (370) then the string is selected (380) and 

preferably the string is stored in the output buffer (385) and 
sent to a target application (390). In accordance with preferred 
embodiments of the present invention when a language model 
series of correlating candidate characters is selected, the series 
15 can be inserted into an output buffer area of the display (116), 
sent to a target application, stored a language model area of the 
display (114), or stored in a language model toolbar option 
(108) to be retrieved upon selection of the toolbar option. 
Referring now to FIGS. 12 & 13, FIG. 12 illustrates 

2 0 graphically a preferred method of character recognition 

interface having an output buffer which can be edited. FIG. 13 
illustrates a flow diagram of operation detailing a preferred 
embodiment of character recognition interface having an 
output buffer that can be edited in accordance with the present 

2 5 invention. In accordance with a preferred embodiment of the 

present invention, a candidate character displayed in the 
output buffer area (116) of the display maybe edited by 
selecting the character through various selection means 
including but not limited to tapping on the character in the 

3 0 output buffer area (116) with a stylus (115) or selecting the 

character via some other input device such as a mouse. 
Preferably, once a candidate character has been selected from 
the output buffer directly for editing, at least one alternative 
candidate character is displayed, and presented, from which 
3 5 selection for replacement of the candidate character can be 
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made. In a preferred embodiment, the alternative candidate 
character is displayed in an alternative candidate character box 
(113) as illustrated in FIG. 10. Optionally, the alternative 
candidate character can be displayed and selected from the 
5 alternative candidate character list area (112). 

Generally, the editing option occurs after a candidate 
character has been displayed in the output buffer area (118) of 
display. Optionally, editing may occur prior to displaying the 
candidate character in the output buffer area (118) of the 
10 display by selecting and displaying on alternative candidate 
character from the candidate character list area (114) of the 
display. 

In a preferred embodiment the input character (102) is 
compared to the recognition data (410). Typically, a first 

15 candidate character representing the graphical image of the 

input character is selected or displayed (420). Preferably, the 
first candidate character is stored in the output buffer (430). 

Typically, a candidate character is identified or selected 
for editing from where it is displayed or stored in the output 

2 0 buffer area of the display (435). As discussed above, various 
options exist to edit a particular candidate character from the 
output buffer area of the display (440). Preferably, the 
candidate character is selected by tapping on the character 
with a stylus or input device. Upon selection of the candidate 

2 5 character, at least one alternative corresponding candidate 

character is presented (445) if generated; preferably it is 
presented in an alternative candidate character box (113) that 
is displayed in close proximity to the candidate character 
selected for editing. If at least one alternative candidate 

3 0 character is generated that represents the graphical image of 

the input data (450) then preferably that alternative candidate 
character is selected (455). The candidate character that was 
displayed or stored in the output buffer for editing is replaced 
by the selected alternative candidate character (460). 
3 5 Preferably, the replacement occurs by simply tapping with a 
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stylus or input device on the selected alternative corresponding 
candidate character displayed in the alternative candidate 
character box. In accordance with the preferred embodiment, 
the selected alternative corresponding candidate character is 
5 then stored or displayed (465), preferably in the output buffer 
area of the display and then sent to the target application 
(470). Optionally, the selected alternative candidate character 
is sent directly to the target application (470). 

If there are no alternative candidate characters that 
1 0 accurately represent the graphical image of the input character 
are generated than various options are provided. If the first 
stored, displayed candidate character in the output buffer is a 
graphic representation of the unrecognized ink of the input 
character (475) then a preferred editing selection is to delete 

1 5 the first stored candidate character (480). Alternatively, if the 

first stored candidate character is not a graphic representation 
of the unrecognized ink of the input character (475) then a 
preferred editing selection is to retrieve the unrecognized 
graphic image of the input character (485), replace the first 

2 0 stored candidate character with the unrecognized graphic 

image of the input character (490), store the unrecognized 
graphic image of the input character in the output buffer (495); 
and send the unrecognized graphic image of the input character 
to the target application (498). 

2 5 Referring now to FIG 12 and FIG. 13, FIG. 12 illustrates 

graphically a preferred method of character recognition 
interface having an on screen keyboard (120). FIG. 13 
illustrates a flow diagram of operation detailing a preferred 
embodiment of character recognition interface having an on 

3 0 screen keyboard in accordance with the present invention. 

Preferably the on screen keyboard is accessed through 
selection of the keyboard option on the toolbar (108). Upon 
selection of the keyboard option, at least one of several various 
keyboards are generated (530) to select from to receive input. 
3 5 Examples of on screen keyboards available in accordance with 
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the preferred embodiments of the present invention include 
but are not limited to the following: English; numeric; symbols; 
pin yin; zhuyin; and other non English languages. Once 
selection of the preferred keyboard is made (540), input 
5 characters may be entered onto the display by tapping the 
graphic representation on the selected keyboard (550). 
Preferably a display location is selected to map the input 
character too (560). Typically the input character is stored 
(570) then mapped to the location selected (580) and sent to 

1 0 the target application. (590) 

Referring now to Figs 14 & 15; and FIGS. 16 & 17, FIGS. 
14 and 16 illustrate graphically a preferred methods of 
character recognition interface having pluralities of user 
adaptable features. FIG. 15, & 17 illustrate a flow diagrams of 
15 14 & 16 respectively, of operations detailing preferred 
embodiments of character recognition interfaces having 
pluralities of user adaptable features in accordance with the 
present invention. In accordance with the preferred 
embodiments of the present invention, various options are 

2 0 available to be selected through accessing the toolbar. 

Examples of such options include setup features, including but 
not limited to the following: pen (input device) (122) width 
(124), color (126), time out option (128), recognition selection 
(130)', and output code selection (132). In accordance with 

2 5 adapting these options for various user preferences, the setup 

feature can be accessed through the tool bar (610, 710) and 
individually selected or selected in pregrouped combinations. 
The figures 14 through 17 illustrate the pregrouped 
combinations, but each option may be selected and processed 

3 0 individually in a similar manner. Preferably the desired set up 

combination is selected (620, 720), the various option or 
options are generated (630, 730), selection of at least one 
option is made (640, 740), input is received for the option by 
means of an input device entering input onto the graphical 
3 5 representation of the option (650, 750). The input is stored 
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(660, 760) and the option selection is closed (670, 770). In a 
preferred embodiment of the present invention, the recognizer 
option (130) of the setup option (122) permits selection of 
recognition data (134) that includes the ideographic Chinese 
characters referred to as GB 1; GB 2, and the Taiwanese 
characters referred to as Big5-1, and Big5-2, In a preferred 
embodiment of the present invention, the output code option 
(132) of the setup option (122) permits selection of the output 
code (132), wherein selection of the output can be made 
between the ideographic Chinese characters referred to as GB 1; 
GB 2, and the Taiwanese characters referred to as Big5-1, and 
Big5-2, Other languages can be easily adapted for the 
Recognition and output code options. 

Referring now to FIGs. 18, 19. and 20. FIG. 18 Illustrates 
an alternative interface in accordance with the present 
invention. FIGS. 19 and 20 illustrate a flow diagram of the 
alternative interface of FIG. 18 in accordance with a preferred 
method of the present invention. 

The alternative preferred embodiment of the present 
invention illustrated in FIGs. 18, 19, and 20 contemplates that 
many users will find it convenient to append a signature or 
other symbols to a document to provide some measure of 
authentication as to the origin of the document. Accordingly, 
upon user selection of the signature icon residing in the 
command area of the primary graphic user interface of FIG. 3, 
the user is presented with the graphic image shown. A 
handwritten personal signature or other symbols entered via 
this interface will be digitized via conventional techniques and 
the signature data representing the handwritten signature used 
as selected by the user. 

As shown in FIG. 18, the signature graphic user interface 
comprises a graphical image 1000 having a menu area 1002, a 
command area (referred to by those skilled in the art as a 
toolbar) 1004, a signature area 1006, and a status area 1008. 
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By comparison of FIG. 18 to FIG. 3, it will be apparent that the 
toolbar 1004 and status area 1008 contain and convey 
different information to the user as appropriate given the 
different functionality offered by the two different graphical 
user interfaces illustrated in FIG. 3 and FIG. 18. 

In the preferred embodiment, the toolbar 1004 of FIG. 18 
offers a variety of command functions by presenting a series of 
icons to provide the functions of: 

New (1010): to open a new signature file; 

Open (1012): to open an existing signature file; 

Save (1014): to save a signature file; 

Copy (1016): to copy of signature file; 

Send (1018): to send the signature file to a selected 

application; . 

Select (1020): to select an application prior to sending the 

signature file; 

Config (1022): to configure some feature or the like; 

About (1024): to inform the user about some feature or 
the like. 

Using these commands, a user is afforded a simple and 
efficient interface to enter a personal signature or other 
symbols to be saved or appended to any of several documents 
prior to storing, printing, or transmitting the document to a 
recipient. In the example of FIG. 18, the signature Earl St 
John" 1026 has been handwritten onto the signature area 1006, 
which can be saved as signature data by selection of the Save 
icon 1014 or perhaps sent to the application designated (via the 
Select icon 1020) to receive the recognized handwritten input 
to be appended thereto as her signature (via the Send icon 
1018). 
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The operational method for the signature graphical user 
interface can be readily understood with reference to FIG. 18. 
PIG. 19, and FIG. 20. Step 1100 of FIG. 19 presents the user 
with the graphical user interface 1000. Typically, the user 
5 would then select to create a new signature document or open 
an existing one. Decision 1102 determines whether the user 
has selected the New icon 1010, and if so, creates a new 
signature file (step 1104) so that the user can handwrite a 
signature or other symbols in the signature area 1006 of the 
1 0 graphical user interface 1000. Conversely, if the user selects 
the Open icon 1012, decision 1106 causes step 1108 to present 
a list of existing signature files to the user for his or her 
selection and opens the selected signature file (step 1110). 

After a new signature file as been created or an existing 

1 5 signature file opened, the user may elect to save the file by 

selecting the Save icon 1014. Decision 1112 detects the user's 
selection of the Save icon and causes the currently active 
signature file to be saved into memory (step 1114) and before 
returning to await the next user command. In a similar 

2 0 manner, the user's selection of the Copy icon 1016 is detected 

by decision 1116 which causes step 1118 to copy the 
information representing whatever handwritten input has been 
entered in the signature area 1006 to be electronically copied 
into memory (commonly referred to as scratchpad memory) as 

2 5 signature data. In this way, the user may copy the signature 

data into other signature files or other applications. 

The signature method of the present invention continues 
in FIG. 20, where decision 1200 determines whether the user 
has selected the Send icon 1018 causing the signature data to 

3 0 be sent to a user selected application (step 1202). To select a 

target application to receive the signature data, the user 
touches the stylus on the Select icon 1020, which is detected by 
decision 1204, and causes a list of potential applications to be 
presented to the user (step 1204), and may include a default 
3 5 selection of the application selected by the user to receive the 
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recognized handwritten input. After user selection of the 
intended application, step 1208 stores information enabling the 
signature data to be sent to the correct application upon the 
user's next selection of the Send icon 1016. 
5 The present invention offers the user a great deal of 

signature or other authentication information via the ConFIG. 
icon 1022. Decision 1210 detects the user's selection of the 
ConFIG. icon 1022 and allows the user to configure (step 1212) 
several parameters such as the color and line width (referred 
10 to as the ink by those skilled in the art) of the presentation of 
the handwritten signature 1026. 

The final command in the preferred embodiment of the 
present invention is the About icon 1024. Upon the user's 
selection of the About icon 1024, decision 1214 causes 

1 5 information to be presented to the user for his or her 

information (step 1216). This information may remain 
displayed until the user closes the information window, or may 
be presented for a time period after which the information 
window automatically closes and the signature method of the 

2 0 present invention awaits the next user instruction. 

Those skilled in the art will find many embodiments of 
the present invention to be useful. One obvious advantage is 
ease of data, or text, input over traditional key-board entry 
methods, including the obvious advantage of the ease of entry 

2 5 of scanned or "off-line" handwritten input into printed data, or 

text Another obvious advantage is recognition of handwritten 
input where the input represents a single character, data point, 
or other unitary identifying graphically representation, that is a 
subset of a large complex set of characters, data points, or other 

3 0 graphical representation. 

It will be apparent to those skilled in the art that the 
disclosed invention may be modified in numerous ways and 
may assume many embodiments other than the preferred 
forms particularly set out and described above. Accordingly, it 
3 5 is intended by the appended claims to cover all modifications of 
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the present invention that fall within the true spirit and scope 
of the present invention and its equivalents. 



What is claimed is: 



WO 97/18526 



PCTYUS96/18220 



-21- 

1. A method comprising the steps of: 

comparing input with recognition data stored in a 
memory to generate at least one candidate character having a 
likelihood of corresponding to said input; 
5 displaying a graphical image having therein a 

graphically writing area from which the input data is accepted 
and a separate output area for presenting at least one 
candidate character; and 

storing in a memory at least one candidate 
0 character for latter retrieval from said memory. 

2. A method of comprising the steps of: 

receiving input from a writing area of a display; 

generating a plurality of candidate characters 
5 having a statistical likelihood of corresponding to said input by 
comparison of said input with recognition data stored in a 
memory; 

displaying said plurality of candidate characters in 
a candidate list area of said display; 

selecting a most likely candidate character from the 
plurality of candidate characters; 

displaying the most likely candidate character in an 
output buffer area of said display. 

3. The method of claim 2, further comprising the step of: 
storing the most likely candidate character in a 

memory. 

4. A method comprising the steps of: 
receiving input data from a writing area of a 

display 

comparing the input data with character recognition 
data to identify at least one likely candidate character likely to 
represent the input data, and identifying at least one 
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alternative candidate character likely to represent the input 
data; 

displaying the at least one likely candidate 
character in an output buffer area of said display; and 
5 displaying the at least one alternative candidate 

character in a candidate list area of said display. 

5. The method of claim 4, further comprising the step of: 

storing the at least one likely candidate character in 

10 a memory. 

6. The method of claim 4, further comprising the step of: 

storing the at least one alternative candidate 
character in a memory. 

l5 7. The method of Claim 4, further comprising the steps of: 

replacing the at least one likely candidate character 

in the output buffer and stored in the memory with one of the 

alternative candidate characters selected from the at least one 

alternative candidate characters; and 
20 displaying the selected alternative candidate 

character in the output buffer. 

8 The method of Claim 4, wherein the at least one 
alternative candidate character can be accessed, displayed, and 

2 5 selected from, by selecting the at least one likely candidate 

character displayed in the output buffer for replacement, 
where the at least one likely candidate character and the at 
least one alternative candidate character were generated to 
represent the same input data. 

30 

9. A method comprising the steps of: 

comparing input data received from a writing area 
of a display with character recognition data; 

identifying at least one likely candidate character 

3 5 representing a word likely to represent the input data; 
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identifying at least one alternative candidate 
character representing an alternative word likely to represent 

the input data; 

identifying at least one series of correlating 
candidate characters for the at least one likely candidate 
character and for each alternative candidate character, each 
series of correlating candidate characters representing a series 
of correlating words likely to include a word representing the 
input data; 

displaying the at least one likely candidate 
character in an output buffer area of said display; 

displaying the each alternative candidate character 
in a candidate list area of said display; and 

displaying the at least one series of correlating 
candidate characters in a correlating candidate characters area 
of the display. 

10 The method of claim 9, further comprising the steps of: 

storing the at least one likely candidate character in 

2 0 a memory. , - 

11 The method of claim 9, further comprising the steps of. 

storing the plurality of alternative candidate 
characters in a memory. 

12 The method of claim 9, further comprising the steps of. 
storing the at least one series of correlating 

candidate characters in a memory. 

13 The method of claim 9, further comprising the steps of: 
replacing the at least one likely candidate character 

in the output buffer with one of the likely series of correlating 
candidate characters selected from the at least one likely series 
of correlating candidate characters displayed in the correlating 
candidate character area of the display; and 

displaying the selected likely series of candidate 

3 5 characters in the output buffer. 



25 



30 
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14. The method of claim 13, wherein the correlating 
candidate characters area of the display is a language model 
area and the likely series of correlating candidate characters is 

5 selected from the at least one likely series of correlating 
candidate characters displayed in the language model area. 

15. The method of claim 13, wherein the correlating 
candidate characters area of the display is displayed through 

1 0 selection of a toolbar language model option and the likely 

series of correlating candidate characters is selected from the at 
least one likely series of correlating candidate characters 
displayed by selecting the tool bar language model option. 

16. The method of claim 9, further comprising the steps of: 
replacing the at least one likely candidate character 

displayed in the output buffer with one of the alternative 

candidate characters selected from the at least one alternative 
candidate characters; 

displaying the selected alternative candidate 

character in the output buffer; 

replacing the alternative candidate character 
displayed in the output buffer and stored in the memory with 
one of the likely series of correlating candidate characters 
selected from the at least one likely series of correlating 
candidate characters; and 

displaying the selected likely series of correlating 
candidate characters in the output buffer. 

3 0 17. The method of claim 16, wherein the correlating 

candidate characters area of the display is displayed in a 
language model area and the likely series of correlating 
candidate characters is selected from the at least one likely 
series of correlating candidate characters displayed in the 

3 5 language model area. 
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18 The method of claim 16, wherein the correlating 
candidate characters area of the display is displayed through 
selection of a toolbar language model option and the likely 

5 series of correlating candidate characters is selected from the at 
least one likely series of correlating candidate characters 
displayed by selecting the tool bar language model option. 

19 A method comprising the steps of: 

, 0 comparing input data received into a writing area of 

a display with character recognition data 

identifying at least one likely candidate character 
representing a word likely to represent the input data; 

identifying at least one likely series of correlating 
1 5 candidate characters for the at least one likely candidate 

character representing a series of correlated words likely to 
include a word representing the input data; 

displaying the at least one likely candidate 
character in an output buffer area of said display; and 
20 displaying the at least one likely series of 

correlating candidate characters in a correlating candidate area 
of the display. 

20 The method of claim 19, further comprising the steps of: 
storing the at least one likely candidate character in 



25 



a memory. 



21 The method of claim 19, further comprising the steps of: 

storing the at least one likely series of correlating 
3 0 candidate characters in a memory. 

22 The method of claim 19, further comprising the steps of: 

replacing the at least one likely candidate character 
in the output buffer and stored in the memory with one of the 
3 5 likely series of correlating candidate characters selected from 
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the at least one likely series of correlating candidate characters 
; and 

displaying the selected likely series of candidate 
characters in the output buffer. 

23. The method of 19., wherein the correlating candidate 
characters area of the display is displayed in a language model 
area and the likely series of correlating candidate characters is 
selected from the at least one likely series of correlating 
candidate characters displayed in the language model area. 

24. The method of claim 19, wherein the correlating 
candidate characters area of the display is displayed through 
selection of a toolbar language model option and the likely 

5 series of correlating candidate characters is selected from the at 
least one likely series of correlating candidate characters 
displayed by selecting the tool bar language model option. 

25. A method comprising the steps of: 

0 comparing input data received from a writing area 

of a display with character recognition data; 

identifying at least one series of correlating 

candidate characters representing a series of correlated words 

likely to include a word representing the input data; and 
5 displaying the at least one series of correlating 

candidate characters in an area of the display. 

26. The method of claim 25, further comprising the steps of: 

storing the at least one series of correlating 
candidate characters in a memory. 

0 

27. The method of claim 25, wherein the at least one series of 
correlating candidate characters is displayed in an output 
buffer area of the display. 
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28. The method of claim 25. wherein the at least one series of 
correlating candidate characters is displayed in language model 
list area of the display. 

5 29. A method comprising the steps of: 

comparing input data received into a writing area of 
a display with character recognition data; 

identifying at least one likely series of correlating 
candidate characters representing a series of correlated words 
10 likely to include a word representing the input data; 

identifying at least one alternative series of 
correlating candidate characters representing an alternative 
series of correlated words likely to include a word representing 
the input data; 

j 5 displaying the at least one series of correlating 

candidate characters in an output buffer area of the display; 

displaying the at least one alternative series of 
correlating candidate characters in a language model area of 
2 0 the display. 

30. The method of claim 29, further comprising the steps of: 
storing the at least one series of correlating 

candidate characters in a memory. 
2 5 31. The method of claim 29, further comprising the steps of: 
storing the at least one alternative series of 

correlating candidate characters in a memory. 

32. The method of claim 29, further comprising the steps of: 
30 ' replacing the at least one likely series of correlating 

candidate characters in the output buffer and stored in the 
memory with one of the alternative series of correlating 
candidate characters selected from the at least one alternative 
series of correlating candidate characters ; and 
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displaying the selected alternative series of 
candidate characters in the output buffer. 

33. A method, comprising the steps of: 
5 displaying a graphical image having an area thereof for 

receiving handwritten characters for comparison with 
recognition data to provide candidate recognition information 
comprising at least one likely candidate character presented in 
an output area of the graphical image, at least one alternate 
10 candidate character presented in a candidate character area of 
the graphical image, and at least one series of correlating 
candidate characters representing at least one set of words or 
phrases likely to be associated with the at least one likely 
candidate character, the series of correlating candidate 
15 characters presented in the correlating candidate character 
area of the graphical image; and 

processing user instructions to replace the at least one 
likely candidate character presented in the output area with a 
candidate character selected from the at least one alternative 
2 0 candidate characters presented in the candidate character area 
of the graphical image, and replacing the at least one series of 
correlating candidate characters in the correlating candidate 
character area with a second series of correlating candidate 
characters likely to be associated with the candidate character 

2 5 selected from the at least one alternative candidate characters. 

34 The method of claim 4, further comprising the steps of: 

providing a tool bar that is movable about a display 
area, said tool bar having a plurality of options from which to 

3 0 select including but not limited to the following: language 

model display; on screen keyboard options; setup options; 
status bar options; and copy, send, clear, and help options. 



35. The method of claim 4, wherein the toolbar may be 
3 5 disabled. 
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36. The method of claim 34, wherein said on screen keyboard 
is displayed when the toolbar on screen option is selected; said 
on screen keyboard having a plurality keyboards from which to 
5 select and enter input data including but not limited to the 
following: English; numeric; symbol; pin yin; and shuyin. 

36. The method of claim 34, wherein each of the plurality of 
options of the tool bar has a prestored default setting; each of 

1 0 said default settings is configured to be replaceable with a 

preferred option input setting. 

37. The method of claim 36, wherein the preferred option 
input setting replaces the prestored default setting and is then 

1 5 stored as the prestored default setting. 

38. A device, comprising: 

a digitizing display, said display having an input display 
area, an output buffer display area, a candidate list display 

2 0 area, and language model display area, said digitizing display 

adapted to receive into the input display area handwritten 
input; 

display processing circuitry for presenting a graphical 
image on the digitizing display representing the handwritten 

2 5 input, and for processing the handwritten input to provide 

handwritten information data; 

a memory having at least recognition data stored therein; 

digital processing circuitry for comparing the handwritten 
information data with the recognition data to provide to the 

3 0 display processing circuitry candidate recognition information 

including at least one likely candidate character representing at 
least one likely word corresponding to the handwritten input; 
at least one likely alternative candidate character representing 
at least one likely alternative word corresponding to the 
3 5 handwritten character, and at least one series of correlating 
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candidate characters representing a series of correlating words 
that include at least one likely word representing the 
handwritten input; 

wherein the display circuitry displays for user interaction 
in an output buffer area the at least one likely candidate 
character, and displays in a candidate list area the at least one 
likely alternative candidate character, and displays in the 
language model display area at least one series of correlating 
candidate characters. 



39. A device, comprising: 

a digitizing display for digitizing into a plurality of 
graphical images that represent input data and for displaying 
the graphical images, said plurality of graphical images 
15 including at least one likely candidate character displayed in 
one portion of the digitizing display, at least one likely 
alternative candidate character displayed in a second portion of 
the digitizing display, and at least one series of correlating 
candidate characters displayed in a third portion of the 

2 0 digitizing display; 

a memory having at least recognition data stored therein; 

digital processing circuitry for processing input data to 
provide candidate character information for comparison with 
the recognition data to provide said plurality of graphical 

2 5 images that represents the input data for user interaction. 

40. A device, comprising: 

a digitizing display for presenting a graphical image on 
the digitizing display and for displaying a likely candidate 

3 0 character in one portion of the graphical image, other candidate 

recognition information in a second portion of the graphical 
image, and characters or words that may be likely characters or 
words' to associate with the likely candidate character in a third 
portion of the graphical image; 
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display processing circuitry for displaying the graphical 
image on the digitizing display and for processing characters 
handwritten within a writing area of the graphical image to 
provide information representing the handwritten characters; 

a memory having instructions and recognition data stored 
therein; 

digital processing circuitry including a digital processing 
device for processing the instructions stored in the memory to 
compare the information representing the handwritten 
characters with the recognition data to provide candidate 
recognition information including at least the likely candidate 
character to the display processing circuitry, and for processing 
the instructions to generate the characters or words that may 
be likely characters or words to associate with the likely 
candidate character. 



41. A device, comprising: 
a digitizing display; 

a memory having instructions and recognition data stored 
therein; 

digital processing circuitry including a programmable 
digital processing device capable of performing a plurality of 
functions by processing at least portions of the instructions 
stored in the memory, the plurality of functions comprising; 

displaying a graphical image on the digitizing 

display; 

processing characters handwritten within a writing 
area of the graphical image to provide information representing 
the handwritten characters; 
) comparing the information representing the 

handwritten characters with the recognition data stored in the 
memory to provide candidate recognition information including 
at least a likely candidate character; 
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generating characters or words that may be likely 
characters or words to associate with the likely candidate 

character; and 

displaying the likely candidate character in one 
portion of the graphical image, other candidate recognition 
information in a second portion of the graphical image, and 
characters or words that may be likely characters or words to 
associate with the likely candidate character in a third portion 
of the graphical image. 



42. A device, comprising: 
a digitizing display; 

a memory having instructions and recognition data stored 
therein; 

1 5 digital processing circuitry including a programmable 

digital processing device capable of emulating the functionality 
of a plurality of circuits by processing at least portions of the 
instructions stored in the memory, the plurality of circuits 
comprising; 

20 display processing circuitry for displaying a 

graphical image on the digitizing display; 

digitizing circuitry for processing characters 

handwritten within a writing area of the graphical image to 

provide information representing the handwritten characters; 
25 " comparison circuitry for comparing the information 

representing the handwritten characters with the recognition 

data stored in the memory to provide candidate recognition 

information including at least a likely candidate character to 

the display processing circuitry; 
30 circuitry to generate characters or words that may 

be likely characters or words to associate with the likely 

candidate character; and 

graphics generation circuitry for generating the 

graphical image and for displaying a likely candidate character 
3 5 in one portion of the graphical image, other candidate 



WO 97/18526 PCT/US96/18220 



-33- 

recognition information in a second portion of the graphical 
image, and characters or words that may be likely characters or 
words to associate with the likely candidate character in a third 
portion of the graphical image. 

5 

43. A device, comprising: 
a digitizing display; 

a memory having an operating system and application 
programs stored therein, and including handwriting recognition 
1 0 instructions and recognition data; 

digital processing circuitry including a programmable 
digital processing device capable of functional emulation of a 
plurality of circuits by processing the handwriting recognition 
instructions in accordance with the operating system, the 

1 5 plurality of circuits comprising; 

display processing circuitry for displaying a 
graphical image on the digitizing display; 

digitizing circuitry for processing characters 
handwritten within a writing area of the graphical image to 

2 0 provide information representing the handwritten characters; 

comparison circuitry for comparing the information 
representing the handwritten characters with the recognition 
data stored in the memory to provide candidate recognition 
information including at least a likely candidate character to 

2 5 the display processing circuitry; 

circuitry to generate characters or words that may 
be likely characters or words to associate with the likely 
candidate character; and 

graphics generation circuitry for generating the 

3 0 graphical image and for displaying a likely candidate character 

in one portion of the graphical image, other candidate 
recognition information in a second portion of the graphical 
image, and characters or words that may be likely characters or 
words' to associate with the likely candidate character in a third 
3 5 portion of the graphical image. 
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44. A device, comprising: 
a digitizing display; 

a memory having an operating system and application 
5 programs stored therein, and including handwriting recognition 
instructions and recognition data; 

digital processing circuitry including a programmable 
digital processing device capable of functional emulation of a 
plurality of circuits by processing the handwriting recognition 

1 0 instructions in accordance with the operating system, the 

plurality of circuits comprising; 

display processing circuitry for displaying a 
graphical image on the digitizing display; 

digitizing circuitry for processing characters 
15 handwritten within a writing area of the graphical image to 

provide information representing the handwritten characters; 

comparison circuitry for comparing the information 
representing the handwritten characters with the recognition 
data stored in the memory to provide candidate recognition 

2 0 information including at least a likely candidate character to 

the display processing circuitry; 

circuitry to generate characters or words that may 
be likely characters or words to associate with the likely 
candidate character; 
25 graphics generation circuitry for generating the 

graphical image and for displaying a likely candidate character 
in a first portion of the graphical image, other candidate 
recognition information in a second portion of the graphical 
image, and characters or words that may be likely characters or 

3 0 words to associate with the likely candidate character in a third 

portion of the graphical image; 

the plurality of circuits further comprising at least one 
circuit selected from the group of circuits consisting of: 

character editing circuitry capable of replacing the 
3 5 likely candidate character with selected candidate character 
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information from the second portion of the graphical image 
responsive to user instructions to present a corrected 
recognition character in the first portion of the graphical image; 

phrase editing circuitry capable of presenting 
5 selected characters or words from the third portion of the 
digitizing display in the first portion of the graphic image 
responsive to user instructions; 

data routing circuitry operating in accordance with 
the operating system for routing all characters presented in the 
0 first portion of the graphic image to at least one of the 

application programs responsive to user instructions selected in 
a command portion of the graphical image. 

45. The device of claim 10, wherein the circuitry to generate 
5 characters or words that may be likely characters or words to 
associate with the likely candidate character further generates 
a second set of characters or words that may be likely 
characters or words to associate with the corrected candidate 
character provided by the character editing circuitry, whereby 
>0 the graphics generation circuitry operates to replace the 

characters or words in the third portion of the graphic image 
with the second set of character or words. 
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